Conventional motion image processing methods and devices, in which characteristics are extracted from time series images obtained by photographing an object in motion and motion information of the object is estimated based on the extracted characteristics, include one method disclosed in Japanese Patent Laid-open Publication No. Hei 6-89342, in which a marker is attached to a finger tip, and another method, in which a glove in a particular color is worn.
Further, as a motion image estimation method, there is available a method in which a magnetic sensor-equipped data glove and so on is worn on a hand, rather than capturing an object image.
Meanwhile, when a conventional technique is applied to a specific device, for example, application to an authentication system when using an automobile, a key equipped with a magnetic sensor, and so on, is necessary.
The above mentioned motion image processing methods and devices have problems as described below.    (1) A method using a marker attached on a finger tip requires marker identification and information on positional relationship among markers.    (2) It is difficult to apply a method using a glove in a particular color to a background containing the same color as that of the glove or to an image expressed in monochrome tones.    (3) A method using a data glove and so on imposes a burden on an operator.
As a technique for tracking an object in time series images to extract motion information thereof, one available technique identifies an image pattern similar to an image pattern of an object, in adjacent image frames by means of correlation matching, so that the motion is extracted as motion of the object (Japanese Patent Laid-open Publication No. Hei 5-197809).
This, however, has a problem such that rotation or size change of an object may result in a change of an image pattern appearing in an image frame, and matching and thus object tracking cannot be achieved properly.
As other methods, techniques (Japanese Patent Laid-open Publications No. Hei 5-12443, No. Hei 8-263666) for tracking a contour of an object, using an energy minimizing method, to extract motion of an object from the obtained contour, are available.
Although these methods can handle object rotation or size change, they cannot handle such a large motion of an object that areas occupied by an object do not overlap between adjacent image frames because the contour in a preceding image frame is used as an initial value in solving an energy minimizing problem.
Other techniques are also available, in which a reference point is set in an area with an object in the initial screen, and the reference point is tracked among adjacent image frames (S. T. Barnard, et al.: “Disparity Analysis of Images, IEEE Transactions on Pattern Analysis and Machine Intelligence”, Vol. PAMI-2, No. 4, July 1980). According to this technique, to track a reference point, candidates for a corresponding point of each reference point are determined through correlation matching using an image pattern in a small area around the reference point, and a corresponding point is then determined among the candidates in consideration of consistency as a whole. This can handle object rotation and size change because a local characteristic of an object is used for tracking.
This can also handle an object with such a large motion that areas occupied by an object do not overlap between adjacent image frames.
In the above conventional techniques, in which object tracking is made based on correspondence of reference points in adjacent image frames, how to determine a reference point is an important matter.
A distinctively characteristic point, such as an angle, may often be used as a reference point in such conventional techniques. Use of such a reference point makes it to obtain a point corresponding to a reference point in adjacent image frames, using, for example, an image pattern in a small area around the reference point.
Although a distinctively characteristic point, such as an angle, is effectively used as a reference point, as described above, the number of such points is limited. Therefore, the upper limit of the number of reference points depends on the shape of an object.
Also, should a focused reference point disappear from the image due to being hidden or for other reasons, the reference point is no longer trackable.
In addition, the shape of an object is limited as these methods cannot handle an object with a free-hand contour or a distinctive feature (such as an angle).
The present invention has been conceived to overcome the above described conventional problems, and aims to provide a motion image processing method, device, and medium for allowing stable tracking of any motion of an object in any shape. The present invention also aims to provide a motion image processing method, device, and medium for calculating motion information of a finger tip, and so on, which draws a pattern in air, adapted to be applied to user authorization when a user attempts to use a specific device (e.g., an automobile).